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Remarks 

Claims 54-85 are currently pending in the subject application and are presently under 
consideration. Claims 54, 55, 57, 58, 60-64, 67, 69, 71, 73, 75, and 81 have been amended as 
shown on pages 2-7 of the Reply. Claims 59, 72, and 80 have been cancelled. Claims 86-102 
have been withdrawn. 

Applicants' representative thanks Examiner Sax for the courtesies extended during the 
telephonic interview conducted on February 24, 2009. The participants discussed possible 
amendments to overcome the rejections under 35 U.S.C §112 regarding collection of passive 
feedback using facial expression input. The claim amendments herein incorporate ideas 
proposed during the interview for overcoming this rejection, including suggestions offered by the 
Examiner. The amended claim features were also discussed in view of the currently cited 
references, and the Examiner indicated that he would perform an additional search on these 
features upon receipt of the Reply. 

Favorable reconsideration of the subject patent application is respectfully requested in 
view of the comments and amendments herein. 

I. Rejection of Claims 54-68, 70-85 Under 35 U.S.C §112 

Claims 54-68, 70-85 stand rejected under 35 U.S.C §112, second paragraph, as allegedly 
being incomplete for omitting essential steps. Independent claims 54 and 71 have been amended 
herein to disclose additional features regarding passive feedback monitoring. It is therefore 
respectfully requested that this rejection be withdrawn. 

II. Rejection of Claim 69 Under 35 U.S.C §112 

Claim 69 stands rejected under 35 U.S.C. § 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. Claim 69 has been amended herein to correct its parent claim reference, thereby 
addressing the antecedent basis issue raised in the Office Action. 

III. Rejection of Claim 69 Under 35 U.S.C. S103(a) 

Claim 69 stands rejected under 35 U.S.C. § 103(a) as being unpatentable over Jain, et ah 
(6567980) and Morris (2002/0088000). It is respectfully submitted that this rejection should be 
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withdrawn for at least the following reasons. Jain, et al. and Morris, individually or in 
combination, do not teach or suggest all aspects set forth in the subject claims. 

A factfinder should be aware, of course, of the distortion caused by hindsight 
bias and must be cautious of arguments reliant upon ex post reasoning. See 

KSR v. Teleflex, 550 U.S. , 127 S. Ct. 1727 (2007) citing Graham v. John 

Deere Co. of Kansas City, 383 U. S. 1, 36 (warning against a "temptation to 
read into the prior art the teachings of the invention in issue" and instructing 
courts to "'guard against slipping into the use of hindsight'" {quoting Monroe 
Auto Equipment Co. v. Heckethorn Mfg. & Supply Co., 332 F. 2d 406, 412 
(CA6 1964))). 

The subject claims relate to authoring and intelligent presentation of media clips. Media 
clips {e.g. pictures, audiovisual clips, etc.) can be stored together with metadata produced for 
each clip, the metadata describing features of the clip such as, but not limited to, a mood or 
theme conveyed by the clip, a person or object identified in the clip, or a date for the clip. The 
metadata for the stored clips can be analyzed to identify clips having a relationship, and a playlist 
of such related clips can be generated for presentation to a user. In particular, amended 
independent claim 54 recites, a metadata analyzer that analyzes the annotating metadata of the 
retrieved scene together with annotating metadata of at least one other scene in the media store 
in order to identify a relationship; [and] a playlist generator that evaluates the relationship and 
produces a playlist of related scenes retrieved from the media store based on the relationship. 

Jain, et al. does not disclose such a technique for creating a playlist of media scenes. 
Jain, et al. relates to a video cataloguing system that extracts metadata from a video stream and 
indexes the video using the extracted metadata. The result of this process is an HTML output 
containing intelligently selected browsable keyframes from the video, displayed with metadata 
tracks for close-caption text, audio classes, speech, keywords, and speakers corresponding with 
each keyframe. However, the cited reference does not teach or suggest identifying relationships 
between scenes based on an analysis of metadata associated with each scene, or building a 
playlist of related scenes based on these relationships. Rather, Jain, et al. is primarily concerned 
with indexing a video stream together with extracted metadata and hyperlinks to facilitate 
manual perusal of the indexed video, but does not contemplate identifying relationships between 
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scenes within the video, much less selecting scenes having such a relationship for inclusion in a 
playlist. 

Morris is also silent regarding these aspects. Morris relates to a method for controlling 
access to metadata associated with an image by assigning user roles to each metadata element, 
such that the metadata can only be accessed by users having an approved role. However, Morris 
makes no mention of creating a playlist of images or scenes, or selecting scenes for such a 
playlist based on an analysis of metadata associated with the respective scenes, as describe 
above. 

The subject claims also disclose that user feedback can be passively monitored during 
playback of the playlist of scenes. This can include monitoring the viewer's emotional reactions, 
mood, or facial expressions as the scenes of the playlist are displayed. Based on the user's 
passive feedback for a given scene , the playlist can be updated by adding or removing similar 
scenes in accordance with the user's monitored reactions. In particular, amended independent 
claim 54 goes on to recite, a viewer that displays the playlist of related scenes; a feedback 
receiver that monitors passive feedback in response to the playlist displayed by the viewer, the 
passive feedback comprises at least a user's emotional reaction as inferred by monitoring the 
user's facial expression using an image capture device and recognizing the user's emotional 
reaction via the monitored facial expressions; and a playlist updater that updates the playlist 
based at least in part on the passive feedback. Contrary to assertions made in the Office Action, 
Jain, et al. does not teach or suggest monitoring a user's emotional reaction in connection with 
viewing a playlist of scenes. Arguing that Jain, et al. discloses these monitoring aspects, the 
Office Action indicates passages relating to extraction of keyframes, timecodes, and textual and 
audio metadata from a video stream. However, these steps do not relate to playback of a playlist 
to a user, but rather to construction of an index for the video stream. Moreover, with regard to 
user feedback, the indicated passages of Jain, et al. disclose only that a user can define 
annotations for a video clip by marking in- and out-points for the clip. This in no way suggests 
monitoring a viewer's emotional reaction, as determined by capturing a user's facial expressions, 
during playback of a video. It therefore cannot be said that the cited reference discloses updating 
such playlist of related scenes in accordance with feedback provided by such monitoring. Morris 
also fails to teach or suggest updating a playlist based on the above-mentioned passive feedback. 
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Similarly, amended independent claim 71 recites, analyzing the metadata associated with 
the scene in view of metadata associated with at least one other scene in the media store; 
identifying at least one relationship between a subset of scenes in the media store based on the 
analysis; generating a playlist comprising the subset of scenes based on the identified 
relationship... recording passive feedback from the user as the playlist is presented, the passive 
feedback comprising at least an inferred emotional reaction by the user as recognized by 
reading the user's facial expressions using an image capture device; and updating the playlist 
based at least on the passive feedback. Neither Jain, et al. nor Morris teach or suggest these 
features, as discussed supra. 

With further regard to determining a relationship between scenes, amended claim 64 
recites, the metadata analyzer computes a similarity value for the relationship between 
respective scenes, while claim 65 recites, the playlist generator produces the playlist of related 
scenes based at least in part on the similarity value. As already noted, neither Jain, et al. nor 
Morris discloses building a playlist of scenes based on a determined relationship between the 
scenes. More specifically, the cited references do not contemplate computing a similarity value 
representing a relationship between scenes. 

Claim 77 discloses additional information regarding this similarity score, reciting, the act 
of evaluating a relationship further comprises computing a similarity score for the relationship 
based upon the metadata that comprises information concerning at least one of a present face, 
a present item, a present voice, a present mood, or a present theme. As discussed above, 
neither cited reference teaches or suggest such a similarity score representing a relationship 
between scenes. The cited references therefore also fail to disclose that such a similarity score 
can be based on the presence of a particular face, item, voice, mood, or theme within the 
respective scenes. 

In view of at least the foregoing, it is respectfully submitted that Jain, et al. and Morris, 
individually or in combination, do not teach or suggest all features of amended independent 
claims 54 and 71 (and all claims depending there from), and as such fail to make obvious the 
present invention. It is therefore requested that this rejection be withdrawn. 
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Conclusion 

The present application is believed to be in condition for allowance in view of the above 
comments and amendments. A prompt action to such end is earnestly solicited. 

In the event any fees are due in connection with this document, the Commissioner is 
authorized to charge those fees to Deposit Account No. 50-1063 [MSFTP304US]. 

Should the Examiner believe a telephone interview would be helpful to expedite 
favorable prosecution, the Examiner is invited to contact applicants' undersigned representative 
at the telephone number below. 



Respectfully submitted, 
Amin, Turocy & Calvin, llp 



/Himanshu S . Amin/ 
Himanshu S. Amin 
Reg. No. 40,894 



Amin, Turocy & Calvin, llp 
127 Public Square 
57 th Floor, Key Tower 
Cleveland, Ohio 44114 
Telephone (216) 696-8730 
Facsimile (216) 696-8731 
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